Triggering items in "More Actions" using API

#1

So I have some Custom Incident Actions and also Jira Server integration, which can be run from the interface. Can I also execute it using API? I looked through the API Explorer, but don’t see this possibility.

Thanks

#2

Hi James,

I don’t think there is a way to do what you’re describing today (trigger an action over the API). Those actions are only triggered manually on a mobile or web applications.

It could be something we consider in the future. Is there a particular situation when you would like to trigger these actions over the API?

Thanks for posting,
John

#3

Hi John, thanks for the response.
With the newly introduced AWS Eventbridge integration, I am looking forward to completely automate some of our incident management processes.
We have an incoming alert, which we know is guaranteed outage, however there has been some false-positives. I was going to write a Lambda that double checks the conditions, and if true, use the Jira integration to open a ticket via API back, which would invocate the incident.annotate webhook to Eventbridge, whic would kick off the entire workflow.

Thanks
James

#4

Hi James,

Thanks for providing the example. Can you confirm that I’m following your desired flow correctly?

  • Monitoring Alert is received in PagerDuty
  • PagerDuty creates an incident as a result of the Alert
  • PagerDuty notifies the on-call
  • On-call will review the Alert and conclude it is an outage
  • On-call presses the Custom Action button to trigger an AWS Lambda function to evaluate the conditions
  • If the AWS conditions are true, then call PagerDuty to create a Jira ticket and sync it to PagerDuty incident

Best,
Wilson

#5

Hi Wilson,
Thanks for the reply. No, here is the desired workflow:

  • Monitoring Alert is received in PagerDuty
  • PagerDuty creates an incident as a result of the Alert
  • Alert is in a Service with Eventbridge extension, and the created incident.triggered webhook fires off an AWS Lambda
  • AWS Lambda evaluates the condition, if false, API back to PD to close incident. If condition = true, send API back to PD to open Jira ticket
  • Jira ticket creation triggers an incident.annotate webhook from the alert to Eventbridge, which fires another AWS Lambda which does multiple things to resolve issue, and also sends API calls to our status page.

Hope this was helpful. Now, this can currently be fully automated if I choose to open a Jira ticket automatically when Alert is created, however like I said, there has been some false-positives. Since I am wanting to use this to include updates to our status page, I wanted to build this extra layer of confirmation. I am looking to take the on-call engineer out of the equation.

Thanks
James

#6

Thanks for the detailed flow. Added some thoughts and responses from the EventBridge point.

Alert is in a Service with Eventbridge extension, and the created incident.triggered webhook fires off an AWS Lambda

  • After the incident is created in PagerDuty, an incident.trigger webhook will be sent to EventBridge. This looks good so far. You may want to tighten the Event Pattern in your Rule to look for specific services, event type, etc…

AWS Lambda evaluates the condition, if false, API back to PD to close the incident. If condition = true, send API back to PD to open Jira ticket

  • Here, if the condition = true, you’re making a PUT /incidents to resolve the incident
  • If the condition = false, you’d like to automatically have PagerDuty open up a Jira ticket (similar to pressing the Create Jira button from the PagerDuty UI. It is possible to call the Create Jira Issue function via API, it is important to note that it IS NOT a publicly supported API endpoint (not designed for public consumption). With that, if you wanted to still move forward knowing that we reserve the right to make changes to that endpoint without notice, you can perform the same function by making a POST /incidents/{incident_id}/custom_action with a body that references the Jira Webhook ID.

Jira ticket creation triggers an incident.annotate webhook from the alert to Eventbridge, which fires another AWS Lambda which does multiple things to resolve issue, and also sends API calls to our status page.

  • To clarify, the incident.annotate webhook in PagerDuty will only be generated and fired if there’s a “Note” added to the Incident. Notes can be added either through Jira Comment and PagerDuty note syncing, or another integration that’s feeding notes into the PagerDuty incident. I’m not sure whether you have some automated workflow in Jira or another tool that adds a note back into the PagerDuty incident to fire the incident.annotate webhook to AWS EventBridge to do the multiple things and update the status page.

Hope that helps!

Best,
Wilson